Makeup support apparatus and method for supporting makeup

ABSTRACT

A makeup support apparatus includes an image capturer that captures an image of a user&#39;s face, a display that displays the captured image, and a guide superimposer that superimposes a guide image upon the displayed image. The guide image indicates a region of the user&#39;s face in which makeup is to be applied.

BACKGROUND

1. Technical Field

The present disclosure relates to a makeup support apparatus and amethod for supporting makeup that support a user in putting on facialmakeup.

2. Description of the Related Art

When putting on makeup, people often use others' makeup as references,such as photographs of models included in fashion magazines. However,because the arrangement, shapes, and sizes of facial components (forexample, eyes, nose, and mouth) vary between individuals, it isdifficult to accurately imitate others' makeup.

Therefore, a technique for displaying a simulated image of a user's faceafter makeup has been disclosed (for example, refer to JapaneseUnexamined Patent Application Publication No. 2007-257194). In thetechnique disclosed in Japanese Unexamined Patent ApplicationPublication No. 2007-257194 (hereinafter referred to as a “techniquedisclosed in an example of the related art”), the simulated image of theuser's face after makeup is generated by superimposing an image ofapplied makeup upon a still image of the user's face. According to thetechnique disclosed in the example of the related art, the user can puton makeup in accordance with the simulated image.

SUMMARY

A simulated image, however, is usually different from an image of theuser's actual face before or during makeup. Therefore, the user mightnot understand how she can achieve the makeup shown in the simulatedimage, and might accordingly put on an entirely different makeup thanintended. For this reason, it is difficult to appropriately support auser in putting on makeup using the technique disclosed in the exampleof the related art.

One non-limiting and exemplary embodiment provides a makeup supportapparatus capable of appropriately supporting the user in putting onmakeup.

In one general aspect, the techniques disclosed here feature a makeupsupport apparatus including an image capturer that captures an image ofa user's face, a display that displays the captured image, and a guidesuperimposer that superimposes a guide image upon the displayed image.The guide image indicates a region of the user's face in which makeup isto be applied.

According to the present disclosure, it is possible to appropriatelysupport the user in putting on makeup.

It should be noted that general or specific embodiments may beimplemented as a system, a method, an integrated circuit, a computerprogram, a storage medium, or any selective combination thereof.

Additional benefits and advantages of the disclosed embodiments willbecome apparent from the specification and drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the specification and drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of the configurationof a makeup support apparatus according to a first embodiment of thepresent disclosure;

FIG. 2 is a block diagram illustrating an example of the configurationof a makeup support apparatus according to a second embodiment of thepresent disclosure;

FIG. 3 is a diagram illustrating an example of the content of makeupinformation according to the second embodiment;

FIG. 4 is a diagram illustrating an example of detection target featurepoints according to the second embodiment;

FIG. 5 is a block diagram illustrating an example of the configurationof a guide superimposing unit according to the second embodiment;

FIG. 6 is a flowchart illustrating an example of operations performed bythe makeup support apparatus according to the second embodiment;

FIGS. 7A to 7D are diagrams illustrating an example of a procedure forgenerating a guide image according to the second embodiment; and

FIGS. 8A and 8B are diagrams illustrating an example of aguide-superimposed mirror image according to the second embodiment.

DETAILED DESCRIPTION

Embodiments of the present disclosure will be described in detailhereinafter with reference to the drawings.

First Embodiment

A first embodiment of the present disclosure is an example of a basicaspect of the present disclosure.

FIG. 1 is a block diagram illustrating an example of the configurationof a makeup support apparatus according to this embodiment.

In FIG. 1, a makeup support apparatus 100 includes an image capture unit130, a display unit 170, and a guide superimposing unit 160.

The image capture unit 130 includes, for example, a lens and an imagingdevice and captures an image of a user's face.

The display unit 170 includes, for example, a display device such as aliquid crystal display and displays a captured image.

The guide superimposing unit 160 superimposes, upon a displayed image, aguide image indicating a region of the user's face in which makeup is tobe applied.

The makeup support apparatus 100 includes, for example, a centralprocessing unit (CPU), a storage medium storing control programs, suchas a read-only memory (ROM), and a working memory such as arandom-access memory (RAM), which are not illustrated. The functions ofthese components are realized by executing the control programs usingthe CPU.

Since the makeup support apparatus 100 can indicate a region of theuser's face in which makeup is to be applied, the makeup supportapparatus 100 can appropriately support the user in putting on makeup.

Second Embodiment

A second embodiment of the present disclosure is an example of aspecific aspect of the present disclosure for supporting the user inputting on makeup by providing a condition in which the user can feel asif she is using a mirror.

Configuration of Makeup Support Apparatus

FIG. 2 is a block diagram illustrating an example of the configurationof a makeup support apparatus 100 according to this embodiment. Themakeup support apparatus 100 is, for example, a tablet terminalincluding a touch panel display and a video camera mounted on the samesurface thereof.

In FIG. 2, the makeup support apparatus 100 includes a storage unit 110,a makeup selection unit 120, an image capture unit 130, an imageobtaining unit 140, a feature point detection unit 150, a guidesuperimposing unit 160, and a display unit 170.

The storage unit 110 stores in advance makeup information correspondingto a plurality of types of makeup. Here, the types of makeup refer tokinds of makeup and include text information indicating impressions of aface after makeup (hereinafter referred to as “made-up face”), such as“grown-up makeup” and “youthful makeup”. The makeup information isinformation defining a region (hereinafter referred to as an“application region”) of a face in which makeup is to be applied as aregion relative to feature points on the face. The feature points on aface (hereinafter referred to as “facial feature points”) are certainfeature (characteristic) points in an image indicating the positions offacial components.

Makeup Information

FIG. 3 is a diagram illustrating an example of the content of the makeupinformation. Here, makeup information defining a region (hereinafterreferred to as an “eye makeup region”) in which makeup is to be appliedto an upper eyelid (hereinafter referred to as “eye makeup”) is to beapplied will be described.

As illustrated in FIG. 3, in the makeup information, an applicationregion 211 is defined as a region relative to, for example, a facialfeature point 212 ₁ at an inner corner of the user's eye, a facialfeature point 212 ₂ at an outer corner of the user's eye, and a facialfeature point 212 ₃ at an inner corner of the user's eyebrow. Theapplication region 211 defined by such makeup information can be usedfor eyelids of various sizes and shapes.

The makeup information may be a template image of an application region.The template image includes reference points corresponding to three ormore certain facial feature points and defines (an outline of) theapplication region as a region relative to the reference points. Thereference points are points set in the template image and refer tocoordinate information at a time when a horizontal x axis and a verticaly axis are provided while determining an upper-left corner of thetemplate image as an origin. The template image may be subjected to anaffine transformation by matching the reference points thereof with thecorresponding facial feature points. In the following description, themakeup information defining an outline of an application region using atemplate image will be referred to as a “template”. A process formatching the reference points with the corresponding facial featurepoints and projecting the application region of a template image onto acaptured image will be referred to as “setting of a template on acaptured image”.

The template is generated by, for example, polygonizing image dataobtained by adding RGB information to a region corresponding to anapplication region through a process for detecting the region and aprocess for extracting a boundary (outline). This generation process maybe performed by the makeup selection unit 120 or the guide superimposingunit 160. The polygonization refers to a process for representing atarget region as a polygon.

The facial feature points 212 ₁ to 212 ₃ hardly move even if the user'seye opens and closes. In the following description, facial featurepoints whose relative positions hardly change due to changes in facialexpression will be referred to as “fixed feature points”. In thisembodiment, the makeup information regarding eye makeup is a templatedefining, using the fixed feature points as reference points, an outlineof the eye makeup region in a state in which the user's eye is closed.

The makeup selection unit 120 illustrated in FIG. 2 refers to thestorage unit 110 and displays the text information corresponding to thetypes of makeup on the touch panel display as options. The makeupselection unit 120 then receives, from the user, an operation forselecting one of the types of makeup. The makeup selection unit 120 neednot receive a selection operation performed by the user but may selectone of the plurality of types of makeup on the basis of certain criteriarelating to the characteristics and age of the user's face. The makeupselection unit 120 then reads makeup information corresponding to theselected type of makeup from the storage unit 110 and outputs the makeupinformation to the guide superimposing unit 160.

The image capture unit 130 includes, for example, a lens and a colorimaging device and captures an image (moving image) of the user's facefrom a position close to the display unit 170, which will be describedlater. The image capture unit 130 is, for example, the above-mentionedvideo camera. The image capture unit 130 sequentially outputs thecaptured moving image to the image obtaining unit 140.

The image obtaining unit 140 obtains a plurality of images included inthe moving image from the input moving image and sequentially outputsimages obtained by horizontally inverting the obtained images to thefeature point detection unit 150 and the guide superimposing unit 160.The inversion of the images (inversion of the moving image) may beperformed by another component, instead. In the following description,images output from the image obtaining unit 140 will be referred to as“captured images”.

The feature point detection unit 150 detects the positions of facialfeature points of the user in each of the captured images input from theimage obtaining unit 140 using a known method for detecting featurepoints in an image, such as pattern matching. The feature pointdetection unit 150 then outputs information (hereinafter referred to as“feature point positional information”) indicating the detectedpositions of the facial feature points in each of the captured images tothe guide superimposing unit 160.

The facial feature points to be detected by the feature point detectionunit 150 (hereinafter referred to as “detection target feature points”)at least include facial feature points referred to by the makeupinformation corresponding to the type of makeup selected by the makeupselection unit 120 (hereinafter referred to as the “selected makeupinformation”). If the selected makeup information defines the eye makeupregion, the detection target feature points at least include facialfeature points (hereinafter referred to as “eye edge feature points”),other than the fixed feature points, located at an edge of the uppereyelid.

Detection Target Feature Points

FIG. 4 is a diagram illustrating an example of the detection targetfeature points. Here, an example in which a type of makeup correspondingto the makeup information defining the eye makeup region has beenselected will be described.

As illustrated in FIG. 4, the detection target feature points includefixed feature points 221 ₁ to 221 ₃ and eye edge feature points 222 ₁ to222 ₃. The detection target feature points also include facial featurepoints 223 ₁ to 223 ₃ located at an edge of a lower eyelid and a facialfeature point 224 at the center of the user's eye.

The fixed feature points 221 ₁ to 221 ₃ make it possible toappropriately set the eye makeup region in the captured image. The eyeedge feature points 222 ₁ to 222 ₃ and the facial feature points 223 ₁to 223 ₃ and 224 make it possible to accurately extract a region(hereinafter referred to as an “eye region”) in which the user's eyeballis exposed.

The guide superimposing unit 160 illustrated in FIG. 2 generates a guideimage indicating an application region on the basis of input featurepoint positional information and makeup information and superimposes thegenerated guide image upon an input captured image. The guidesuperimposing unit 160 outputs an image (hereinafter referred to as an“image obtained as a result of superimposition”) obtained bysuperimposing the guide image upon the captured image to the displayunit 170.

The guide image indicates a region of the user's actual face in whichmakeup is to be applied. The guide image may be an image including linesindicating an outline of the application region, a hatching image, or atranslucent image that is relatively transparent, in order not tointerfere with display of the actual state of each region of the user'sface.

The display unit 170 includes, for example, a liquid crystal display anddisplays an input image obtained as a result of superimposition. Thedisplay unit 170 is, for example, the above-mentioned touch paneldisplay. As described above, the display unit 170 is arranged close tothe image capture unit 130, and displays the image obtained as a resultof superimposition for a user who is looking at the image capture unit130. A captured image included in the image obtained as a result ofsuperimposition has been horizontally inverted.

The image obtained as a result of superimposition is generated anddisplayed in substantially real-time for each image included in a movingimage output from the image capture unit 130. Therefore, a plurality ofimages, which have been obtained as a result of superimposition,displayed by the display unit 170 serve as a moving image (hereinafterreferred to as a “guide-superimposed mirror image”) obtained bysuperimposing guide images that follow the movement of the facialfeature points upon a mirror image of the original moving image.

The makeup support apparatus 100 includes, for example, a CPU, a storagemedium storing control programs, such as a ROM, and a working memorysuch as a RAM, which are not illustrated. In this case, the functions ofthe above-described components are realized by executing the controlprograms using the CPU.

The makeup support apparatus 100 can display, like a mirror, ahorizontally inverted image of the user's face and superimpose guideimages indicating a region in which makeup is to be applied upon thedisplayed image of the user's face.

Details of Guide Superimposing Unit

As described above, in this embodiment, the makeup information regardingeye makeup defines, using the fixed feature points around the user'seyelid as the reference points, the eye makeup region corresponding to astate in which the user's eye is closed. In the following description,the makeup information defining the eye makeup region will be referredto as a “template for eye makeup”.

By using such a template, it becomes possible to perform a process forsetting application regions, including the eye makeup region, in amoving image with a small amount of calculation and keep incorrect guideimages from being displayed more certainly. On the other hand, when theuser's eye is open, the template for eye makeup does not match theactual shape of the user's eyelid, and the guide-superimposed mirrorimage becomes unnatural.

Therefore, in the template for eye makeup, the guide superimposing unit160 detects the eye region in the captured image and performs a processfor removing the detected eye region from the application region definedby the template.

FIG. 5 is a block diagram illustrating an example of the configurationof the guide superimposing unit 160.

In FIG. 5, the guide superimposing unit 160 includes a template settingsection 161, an eye region obtaining section 162, a superimpositiondetermination section 163, a region correction section 164, and a guideimage generation section 165.

The template setting section 161 sets a template (makeup information) ina captured image by matching the positions of fixed feature pointsreferred to by the template (makeup information) with the positions offixed feature points in the captured image. The template setting section161 then outputs, to the guide image generation section 165, information(hereinafter referred to as “region information”) indicating anapplication region (hereinafter referred as a “region to be presented”)determined by the set template.

In the case of a template for eye makeup, however, the template settingsection 161 determines a region defined by the set template as atemporary eye makeup region. The template setting section 161 thenoutputs information indicating the temporary eye makeup region to thesuperimposition determination section 163, not to the guide imagegeneration section 165.

The eye region obtaining section 162 obtains an eye region in thecaptured image on the basis of facial feature points detected from thecaptured image. The eye region obtaining section 162 then outputsinformation indicating the obtained eye region to the superimpositiondetermination section 163.

Such facial feature points include, for example, the eye edge featurepoints 222 ₁ to 222 ₃ and the facial feature points 223 ₁ to 223 ₃ and224 illustrated in FIG. 4. In this case, the eye region obtainingsection 162 calculates, for example, a line connecting the fixed featurepoints 221 ₁ and 221 ₂ and the eye edge feature points 222 ₁ to 222 ₃and a line connecting the fixed feature points 221 ₁ and 221 ₂ and thefacial feature points 223 ₁ to 223 ₃. The eye region obtaining section162 then polygonizes a region (a region in which the facial featurepoint 224 is located) surrounded by these two lines and determines theregion as an eye region. The eye region obtaining section 162 desirablyperforms a known curving process such as spline interpolation on each ofthe two lines.

The superimposition determination section 163 determines, on the basisof the input information, a region (hereinafter referred to as an“eye-superimposed region”) in the temporary eye makeup region in whichan eye region is superimposed. More specifically, for example, thesuperimposition determination section 163 determines, as theeye-superimposed region, a region in which the temporary applicationregion (polygon region) indicated by the template for eye makeup and theeye region (polygon region) overlap. The superimposition determinationsection 163 then outputs information indicating the determinedeye-superimposed region and information indicating the input temporaryeye makeup region to the region correction section 164.

The region correction section 164 calculates a region obtained byremoving the eye-superimposed region from the temporary eye makeupregion on the basis of the input information. That is, the regioncorrection section 164 corrects the eye makeup region on the basis ofthe eye region. The region correction section 164 then outputsinformation indicating the calculated eye makeup region (the region tobe presented) to the guide image generation section 165 as regioninformation regarding eye makeup.

The guide image generation section 165 generates an image to besuperimposed indicating an outline of the region to be presented on thebasis of the input region information and superimposes the image uponthe captured image. That is, the guide image generation section 165sequentially generates a guide-superimposed mirror image and outputs theguide-superimposed mirror image to the display unit 170.

Since the makeup support apparatus 100 includes such a guidesuperimposing unit 160, the makeup support apparatus 100 can easilygenerate and display guide images that accurately follow opening andclosing of the user's eyelid.

Operations Performed by Makeup Support Apparatus

Next, operations performed by the makeup support apparatus 100 will bedescribed.

FIG. 6 is a flowchart illustrating an example of the operationsperformed by the makeup support apparatus 100. FIGS. 7A to 7D arediagrams illustrating an example of a procedure for generating a guideimage using a template for eye makeup and correspond to FIGS. 3 and 4.The same feature points as those illustrated in FIGS. 3 and 4 are giventhe same reference numerals, and description thereof is omitted.

First, in step S1100 illustrated in FIG. 6, the makeup selection unit120 selects a type of makeup. The makeup selection unit 120 then outputsmakeup information corresponding to the selected type of makeup to theguide superimposing unit 160. Here, makeup information including atemplate for eye makeup is output to the guide superimposing unit 160.

In step S1200, the image capture unit 130 begins to capture a movingimage of the user's face. The image capture unit 130 then sequentiallyoutputs the captured moving image to the image obtaining unit 140.

In step S1300, the image obtaining unit 140 obtains an image from theinput moving image. The image obtaining unit 140 then horizontallyinverts the obtained image and outputs the image to the feature pointdetection unit 150 and the guide superimposing unit 160.

In step S1400, the feature point detection unit 150 detects facialfeature points in the input image. The feature point detection unit 150then outputs feature point positional information regarding the detectedfacial feature points to the guide superimposing unit 160.Alternatively, the feature point detection unit 150 may receive makeupinformation from the makeup selection unit 120 and detect only facialfeature points necessary to generate a guide image.

In step S1500, the template setting section 161 of the guidesuperimposing unit 160 sets the template of makeup in the captured imageon the basis of the input makeup information and feature pointpositional information. The template setting section 161 then outputsregion information indicating an application region determined by theset template to the guide image generation section 165.

In the case of the template for eye makeup, however, the templatesetting section 161 sets, as illustrated in FIGS. 7A and 7B, thetemplate in each of a captured image 310 ₁ of the user's face in whichthe user's eye is closed and a captured image 310 ₂ of the user's facein which the user's eye is open, and temporarily determines theapplication region 211. Although, however, the application region 211based on the template for eye makeup seems natural in the captured image310 ₁ of the user's face in which the user's eye is closed, theapplication region 211 seems unnatural in the captured image 310 ₂ ofthe user's face in which the user's eye is open as illustrated in FIG.7B.

Therefore, in the case of eye makeup, the template setting section 161outputs information indicating a temporary application region determinedby the set template to the superimposition determination section 163.

In step S1600 illustrated in FIG. 6, the eye region obtaining section162 of the guide superimposing unit 160 obtains an eye region of thecaptured image on the basis of the input feature point positionalinformation. The eye region obtaining section 162 then outputsinformation indicating the obtained eye region to the superimpositiondetermination section 163.

For example, the eye region obtaining section 162 obtains, asillustrated in FIG. 7C, an eye region 320 on the basis of the fixedfeature points 221 ₁ and 221 ₂, the eye edge feature points 222 ₁ to 222₃, and the facial feature points 223 ₁ to 223 ₃.

In step S1700 illustrated in FIG. 6, the superimposition determinationsection 163 of the guide superimposing unit 160 determines aneye-superimposed region, which is a portion in which an eye makeupregion and an eye region overlap, on the basis of the input information.

The superimposition determination section 163 obtains, for example, aportion in which the application region 211 illustrated in FIG. 7B andthe eye region 320 illustrated in FIG. 7C overlap as theeye-superimposed region.

In step S1800 illustrated in FIG. 6, the region correction section 164of the guide superimposing unit 160 determines a region obtained byremoving the eye-superimposed region from the application region 211,which is the region defined by the template, as an application region(hereinafter referred to as a “region to be presented”) to be presented.The region correction section 164 then outputs the determined region tobe presented for eye makeup to the guide image generation section 165.

For example, the region correction section 164 determines, asillustrated in FIG. 7D, a region 330 in the application region 211illustrated in FIG. 7B in which the eye region 320 illustrated in FIG.7C does not overlap as the region to be presented.

The guide superimposing unit 160 does not perform the processing insteps S1600 to S1800 for makeup other than eye makeup, such as cheekmakeup or eyebrow makeup.

In step S1900 illustrated in FIG. 6, the guide image generation section165 of the guide superimposing unit 160 generates a guide imageindicating an outline of the input region to be presented and generatesa guide-superimposed mirror image by superimposing (combining) the guideimage upon the captured image. The guide image generation section 165then outputs the generated guide-superimposed mirror image to thedisplay unit 170, which displays the guide-superimposed mirror image.

In step S2000, the image obtaining unit 140 determines whether aninstruction to end the process has been issued through a user operationor the like. If an instruction to end the process has not been issued(NO in S2000), the image obtaining unit 140 returns the process to stepS1300. If an instruction to end the process has been issued (YES inS2000), the image obtaining unit 140 ends the process.

FIGS. 8A and 8B are diagrams illustrating an example of theguide-superimposed mirror image and correspond to FIGS. 7A to 7D.

As illustrated in FIGS. 8A and 8B, a guide image 341 is, for example, animage indicating the outline of the region to be presented using whitebroken lines. The guide image 341 suits the shape of the user's eyelidin each of a guide-superimposed mirror image 340 ₁ based on the capturedimage 310 ₁ of the user's face in which the user's eye is closed and aguide-superimposed mirror image 340 ₂ based on the captured image 310 ₂of the user's face in which the user's eye is open. Thus, the guideimage 340 follows the movement of the eyelid.

Time taken until the guide-superimposed mirror image is displayed aftera corresponding original image is captured can be reduced in accordancewith the processing speed of a CPU used, the resolution of the image,the content of the makeup information, and the like. In this case, thedisplayed guide-superimposed mirror image follows the movement of theuser in substantially real-time. The components from the image obtainingunit 140 to the display unit 170 perform stream data processing on themoving image output from the image capture unit 130. Therefore, the usercan put on makeup while feeling as if she is using a mirror.

Advantageous Effects Produced by this Embodiment

As described above, the makeup support apparatus 100 according to thisembodiment can display the user's face while horizontally inverting theuser's face in an image like a mirror. At the same time, the makeupsupport apparatus 100 can superimpose guide images indicating anapplication region upon the user's face in the image. The guide imageshardly interfere with display of the actual state of each region of theuser's face.

By using the makeup support apparatus 100, the user can put on makeupwhile referring to the guide images, which indicate the applicationregion, that follow the movement of the user's face and feeling as ifshe is using a mirror.

As described above, in the example of the related art, only a result ofmakeup is provided, and accordingly the user might undesirably put on anentirely different makeup than intended. In contrast, the makeup supportapparatus 100 displays not a result of makeup but information forguiding the user in a procedure of makeup. As a result, the makeupsupport apparatus 100 can enable the user to put on a makeup that ismore similar to an intended one.

Furthermore, in the example of the related art, since the user needs toput on makeup while seeing both a mirror and a simulated image, the usermight recognize the location of each region of her face differentlybetween in the mirror and in the simulated image, thereby undesirablyapplying makeup to an incorrect region. In contrast, since the makeupsupport apparatus 100 can display the guide-superimposed mirror image infront of the user's face, the user need not see both a mirror and asimulated image. As a result, the makeup support apparatus 100 canenable the user to apply makeup to a correct region.

In addition, if makeup information that defines a region in which eyemakeup is to be applied on the basis of the positions of facial featurepoints on an edge of the user's eye, a complex transformation processmight be necessary in which an image close to the user's eyelid needs tobe divided into a plurality of regions or more facial feature pointsneed to be used. The makeup support apparatus 100 according to thisembodiment, however, determines a region to be presented for eye makeupby calculating a temporary application region using a template for eyemakeup in a state in which the user's eye is closed and performing aprocess for removing an eye region from the temporary applicationregion. Therefore, the makeup support apparatus 100 according to thisembodiment can generate and display accurate guide images for eye makeupwith a small amount of calculation.

The makeup support apparatus 100 can also display guide images forvarious types of makeup other than eye makeup, such as lip, cheek, andeyebrow makeup. The makeup support apparatus 100 may use theabove-described template for makeup other than eye makeup. Furthermore,the makeup support apparatus 100 can simultaneously generate guideimages for a plurality of types of makeup, such as eye makeup and lipmakeup, and superimpose the guide images for the plurality of types ofmakeup upon a single captured image. That is, the makeup supportapparatus 100 can display the guide-superimposed mirror image for makeupof the entirety of the user's face.

In addition, an application region defined by makeup information mayinclude a region shared by a plurality of different types of makeup suchas eye shadow makeup and eyeliner makeup. In addition, when a simulatedimage of makeup is displayed, the makeup support apparatus 100 mayselect color processing in an overlap region from either overlapprocessing or non-overlap processing. Here, the overlap processingrefers to processing in which colors of different types of makeup arecombined with each other, and the non-overlap processing refers toprocessing in which only a color of one of the different types of makeup(a type of makeup applied later in a normal makeup procedure) isdisplayed. In this case, too, a template may be prepared for each of thedifferent types of makeup.

In addition, the makeup support apparatus 100 may perform, during makeupin which the overlap processing is used, the same process fordetermining an overlap between regions that is performed when an overlapbetween a region in which eye makeup is to be applied and an eye regionis determined.

For example, in cheek makeup in which a thick color is applied at thecenter of a pale color, the makeup support apparatus 100 prepares atemplate of a large, round application region and a template of a small,round application region. In the case of overlap makeup, the makeupsupport apparatus 100 displays a small, ground guide image afterdisplaying a large, round guide image.

On the other hand, in the case of non-overlap makeup, the makeup supportapparatus 100 displays a donut-shaped guide image in which a smallcircle is arranged inside a large circle. In this case, the makeupsupport apparatus 100 may determine a region in which applicationregions of the two templates overlap and generates the donut-shapedguide image by removing the overlap region from the large, roundapplication region.

That is, the makeup support apparatus 100 can generate various guideimages with a smaller amount of data and a smaller amount of processingby using template images and performing, as necessary, the process fordetermining an overlap between regions.

In addition, makeup information need not necessarily be a template. Forexample, because the color of a lip region is significantly differentfrom the color of skin around the lip region, an outline of the lipregion can be easily and accurately extracted on the basis of luminanceinformation regarding a captured image. Therefore, if the type of makeupselected is one in which the user puts on lipstick in the lip region,makeup information may define the lip region as the application region.

In addition, although a case in which a guide image hardly interfereswith the actual state of each region of the user's face has beendescribed in this embodiment, the mode of a guide image is not limitedto this. For example, the guide superimposing unit 160 may generate asimulated image of eye makeup by superimposing, upon a captured image,an image of an eye makeup region in which makeup has already beenapplied. If a region to be presented follows opening and closing of theuser's eye as illustrated in FIGS. 8A and 8B, a simulated image thatmore accurately shows the user's face after makeup can be generated thanin the example of the related art, in which a region to be presenteddoes not follow opening and closing of the user's eye. Accordingly, anintended makeup can be achieved more easily.

In addition, types of makeup supported by the makeup support apparatus100 may include not only normal types of makeup but also skin care suchas application of facial serum and massage. In this case, it isdifficult to understand how to take care of the user's skin only on thebasis of a simulated image of the user's face after skin care. However,the makeup support apparatus 100 can appropriately support makeupincluding skin care by displaying guide images indicating a region inwhich the user's skin is to be taken care of.

A makeup support apparatus in the present disclosure includes an imagecapturer that captures an image of a user's face, a display thatdisplays the captured image, and a guide superimposer that superimposesa guide image upon the displayed image. The guide image indicates aregion of the user's face in which makeup is to be applied.

In the makeup support apparatus, the image capturer may capture a movingimage of the user's face. The display may display a moving imageobtained by horizontally inverting the captured moving image. The guidesuperimposer may superimpose a guide image upon each of a plurality ofimages included in the displayed moving image.

In the makeup support apparatus, the guide image may be an imageincluding lines indicating an outline of the region.

The makeup support apparatus may further include a feature pointdetector that detects positions of feature points of the user's face inthe image. The guide superimposer may generate the guide image on thebasis of both makeup information defining the region relatively to thefeature points of the user's face and the detected positions of thefeature points.

In the makeup support apparatus, the region may include a region inwhich eye makeup is to be applied. The feature points may include afeature point located on an edge of an upper eyelid.

In the makeup support apparatus, the makeup information may define theregion in which eye makeup is to be applied in a state in which theuser's eye is closed on the basis of a feature point that does not movewhen the user's eye is opened or closed. The makeup support apparatusmay further include an eye region obtainer that obtains an eye region inthe image. The guide superimposer may superimpose a guide imageindicating a region obtained by removing the obtained eye region fromthe region in which eye makeup is to be applied.

In the makeup support apparatus, the guide superimposer may superimpose,upon the image, an image of makeup applied in the region in which eyemakeup is to be applied.

The makeup support apparatus may further include a storage that storesthe makeup information defining the region for each of a plurality oftypes of makeup, and a makeup selector that selects at least one of theplurality of types of makeup. The guide superimposer may superimpose theguide image on the basis of the makeup information for the selected typeof makeup.

A method for supporting makeup in the present disclosure includes thesteps of capturing an image of a user's face, superimposing, upon thecaptured image, a guide image indicating a region of the user's face inwhich makeup is to be applied, and displaying the image upon which theguide image has been superimposed.

The present disclosure is effective as a makeup support apparatus and amethod for supporting makeup that appropriately support a user inputting on makeup.

What is claimed is:
 1. A makeup support apparatus comprising: an imagecapturer that captures an image of a user's face; a display thatdisplays the captured image; and a guide superimposer that superimposesa guide image upon the displayed image, wherein the guide imageindicates a region of the user's face in which makeup is to be applied.2. The makeup support apparatus according to claim 1, wherein the imagecapturer captures a moving image of the user's face, wherein the displaydisplays a moving image obtained by horizontally inverting the capturedmoving image, and wherein the guide superimposer superimposes a guideimage upon each of a plurality of images included in the displayedmoving image.
 3. The makeup support apparatus according to claim 1,wherein the guide image is an image including lines indicating anoutline of the region.
 4. The makeup support apparatus according toclaim 1, further comprising: a feature point detector that detectspositions of feature points of the user's face in the image, wherein theguide superimposer generates the guide image on the basis of both makeupinformation defining the region relatively to the feature points of theuser's face and the detected positions of the feature points.
 5. Themakeup support apparatus according to claim 4, wherein the regionincludes a region in which eye makeup is to be applied, and wherein thefeature points include a feature point located on an edge of an uppereyelid.
 6. The makeup support apparatus according to claim 5, whereinthe makeup information defines the region in which eye makeup is to beapplied in a state in which the user's eye is closed on the basis of afeature point that does not move when the user's eye is opened orclosed, the makeup support apparatus further comprising: an eye regionobtainer that obtains an eye region in the image, wherein the guidesuperimposer superimposes a guide image indicating a region obtained byremoving the obtained eye region from the region in which eye makeup isto be applied.
 7. The makeup support apparatus according to claim 6,wherein the guide superimposer superimposes, upon the image, an image ofmakeup applied in the region in which eye makeup is to be applied. 8.The makeup support apparatus according to claim 4, further comprising: astorage that stores the makeup information defining the region for eachof a plurality of types of makeup; and a makeup selector that selects atleast one of the plurality of types of makeup, wherein the guidesuperimposer superimposes the guide image on the basis of the makeupinformation for the selected type of makeup.
 9. A method for supportingmakeup, the method comprising the steps of: capturing an image of auser's face; superimposing, upon the captured image, a guide imageindicating a region of the user's face in which makeup is to be applied;and displaying the image upon which the guide image has beensuperimposed.